'Mind the gap': evaluating user physiological response for multi-genre video summarisation

Existing video summarisation techniques are often only capable of summarising video from pre-specified content genres and are often not able to produce personalised summaries as they are not able to source relevant user specific data. Because users often experience strong emotions and associated physiological responses whilst watching video, their physiological response to video content may serve as a new and valuable data source for video summarisation. Previously, we developed the Entertainment-Led VIdeo Summarisation (ELVIS) technique that summarises video based on five physiological response measures: electro-dermal response (EDR), heart rate (HR), blood volume pulse (BVP), respiration rate (RR), and respiration amplitude (RA). Here, we report a statistical analysis on a range of data collected from ELVIS in trials with 100 users relating to five distinct video content genres (Action, Drama, Romance, Horror and Comedy). The results show that the ELVIS, EDR, HR, BVP, RR and RA video summaries all consistently match with the most entertaining video sub-segments as self-reported by the user, and that the composite ELVIS video summaries achieve significantly higher level of overlap compared with a RANDOM selection. More generally, users reported that, compared with video summaries produced by another contemporary video summarisation technique, ELVIS video summaries are comparatively ‘enjoyable’ and ‘informative’ for all five video content genres. We therefore conclude that video summarisation according to users’ physiological responses has great value for future development of video summarisation techniques that are applicable across a wide range of video